Probability Theory

• Probability measure \(\mathrm{P}\) is a measure over \(\Omega\), with:
  • \(\mathrm{P}: \sigma(\Omega) \to [0, 1]\), with \(\mathrm{P}(\varnothing) = 0\) and \(\mathrm{P}(\Omega) = 1\).
  • Countable Additivity: \[ \mathrm{P}\left(\bigcup_{i\in \mathbb{N}}E_i\right) = \sum_{i\in \mathbb{N}}\mathrm{P}({E_i}) \] where \(\{E_i\}\) are pairwise disjoint sets.
• The validity of this definition of a probability measure is precisely given by the Kolmogorov axioms.¹
  1. Non-negativity
  2. Unit measure
  3. σ-additivity

• Probability that a random variable \(X\) takes a value in a measurable set \(S\subseteq E\) is written as \[ \mathrm{P}[X\in S] := \mathrm{P}(\{\omega \in \Omega\mid X(\omega)\in S\}). \]

• 왜도

• 첨도

• Probability function whose domain has infinite elements.
• Random variable \(X\) is absolutely continuous, if there exists a function \(f_X\) such that for each interval \([a,b] \in \mathbb{R}\): \[ \mathrm{P}[a\le X \le b] = \int_a^b f_X(x)\,dx \]

• The probability density function of a random variable \(X\) that takes values in a measure space \((E, \Sigma, \mu)\) is the Radon-Nikodym derivative of the pushforward measure \(X_*\mathrm{P}\) with respect to the reference measure \(\mu\): \[ f_X(x) := \frac{d X_*\mathrm{P}}{d\mu} \]
• That is, \[ X_*\mathrm{P}(A) = \int_A f_X\,d\mu \] for any measurable set \(A\in \Sigma\).

• For a real-valued random variable, it is absolutely continuous univariate distribution satisfying:
  • \[ f_X(x) = \frac{dF_X}{dx}, \]

• \[ f_{X+Y}(x) = (f_X * f_Y)(x) \]
• where \(*\) is the convolution.
• List of convolutions of probability distributions - Wikipedia

• \[ f_{XY} = \int_{-\infty}^\infty f_{X,Y}(x, z/x)\frac{1}{|x|}\,dx \]
• Distribution of the product of two random variables - Wikipedia

• \[ f_{X/Y} = \int_{-\infty}^\infty f_{X,Y}(zy, y)|y|\,dy \]
• Ratio distribution - Wikipedia

\[ [X]_t = \lim_{\Vert P\Vert \to 0} \sum_{k=1}^n(X_{t_k} - X_{t_{k-1}})^2 \] where \(P\) is the partition over the interval \([0, t]\).

• Cross-Variance

A discrete-time stochastic process \( X \) is martingale if, for any time \(n\):

• \[ \operatorname{E}[|X_n|] < \infty, \]
• \[ \operatorname{E}[X_{n+1}\mid X_1,\dots,X_n ] = X_n. \]

Generally, a stochastic process \(Y: T\times \Omega \to S\), where \(S\) is a Banach space with norm, is a martingale with respect to filtration \(\Sigma_{*}\), and probability measure \(\mathrm{P}\), if:

• \( \Sigma_{*} \) is a filtration of the event space \(\Sigma\) of the underlying probability space \((\Omega,\Sigma, \mathrm{P})\).
• \(Y\) is adapted to the filtration \(\mathcal{F}\), that is, for each \(t\) in the index set \(T\), the random variable \(Y_t\) is a \( \Sigma_t \)-measurable function.
• For each \(t\), \(Y_t\) lies in the \(L^p\) space \(L^1(\Omega, \mathcal{F}_t, \mathrm{P}; S)\): \[ \operatorname{E}[\Vert Y_t \Vert_S] < \infty. \]
• For all \(s\) and \(t\) with \(s

A \( \Sigma_{*} \)-adapted stochastic process \( X \) is called an \( \Sigma_* \)-local martingale, if there exists a sequence of \( \Sigma_* \)-stopping times \( \tau_k\colon \Omega \to [0, \infty) \) such that

• the \( \tau_k \) are almost surely increasing: \( \mathrm{P}[\tau_k < \tau_{k+1}] = 1 \),
• the \( \tau_k \) diverge almost surely: \( \mathrm{P}[ \lim_{k\to \infty} \tau_k = \infty ] = 1 \),
• the stopped process \( X_t^{\tau_k} := X_{\min\{t, \tau_k\}} \) is an \( \Sigma_{*} \)-martingale for every \( k \).

Each \( \tau_k \) indicates a single stopping criterion, for example, "if the value of the stochastic process drops below zero".

A real-valued stochastic process \( X \) is called a semimartingale, if it can be decomposed into a local martingale \( M \) and a cádlág adapted process \( A \) of locally bounded variation: \[ X_t = M_t + A_t. \]

• Scaling limit of a random walk. This is the result of the Math/Donsker's Theorem.

• It describes the Brownian motion.
• It is the integral of the white noise generalized by the Gaussian process

A continuous stochastic process \( \{ X_t ; t\in T \}\) is Gaussian if and only if for every finite set of indices \( t_1, \ldots, t_k \) in the index set \( T \) \[ \mathbf{X}_{t_1,\dots, t_k} = (X_{t_1}, \dots, X_{t_k}) \] is a multivariate Gaussian random variable.

The second-order statistics completely defines a Gaussian process.

The variances and covariances can be given by the covariance function \( K(x,x') \), and together with the restriction of the domain, they completely determines the probability density over functions with a continuous domain.

• Gaussian Process Regression

The prior distribution is determined by suitable choice of hyperparameters for the covariance function (or kernel), and the observation is used to update the prior.

It is a form of Bayesian inference, using the Gaussian process as a prior probability distribution.